Service provider optimization of content management

ABSTRACT

A system and method for monitoring the performance associated with fulfilling resource requests and determining optimizations for improving such performance are provided. A processing device obtains and processes performance metric information associated with processing a request for an original resource and any embedded resource. The processing device uses the processed performance metric information to determine a set of service providers for alternatively hosting, processing, and/or transmitting at least a portion of the original resource and/or any embedded resources. In some embodiments, in making such a determination, the processing device assesses performance metric information collected and associated with subsequent resource requests for the original resource and any embedded resources using each of a variety of alternative set(s) of service providers. Aspects of systems and methods for generating recommendations associated with monitoring the operation and performance of service providers with respect to specific resource requests are also provided.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/152,860, now U.S. Pat. No. 9,071,502, titled SERVICE PROVIDEROPTIMIZATION OF CONTENT MANAGEMENT, and filed Jan. 10, 2014, which inturn is a continuation of U.S. patent application Ser. No. 13/620,837,now U.S. Pat. No. 8,631,129, titled SERVICE PROVIDER OPTIMIZATION OFCONTENT MANAGEMENT, and filed Sep. 15, 2012, which in turn is acontinuation of U.S. patent application Ser. No. 13/210,326, now U.S.Pat. No. 8,307,078, titled SERVICE PROVIDER OPTIMIZATION OF CONTENTMANAGEMENT, and filed Aug. 15, 2011, which in turn is a continuation ofU.S. patent application Ser. No. 12/240,740, now U.S. Pat. No.8,051,166, entitled SERVICE PROVIDER OPTIMIZATION OF CONTENT MANAGEMENTand filed Sep. 29, 2008, the disclosures of which are incorporatedherein by reference.

BACKGROUND

Generally described, computing devices and communication networks can beutilized to exchange information. In a common application, a computingdevice can request content from another computing device via acommunication network. For example, a user at a personal computingdevice can utilize a browser application to request a Web page from aserver computing device via the Internet. In such embodiments, the usercomputing device can be referred to as a client computing device and theserver computing device can be referred to as a content provider.

Content providers are generally motivated to provide requested contentto client computing devices often with consideration of efficienttransmission of the requested content to the client computing deviceand/or consideration of a cost associated with the transmission of thecontent. For larger scale implementations, a content provider mayreceive content requests from a high volume of client computing deviceswhich can place a strain on the content provider's computing resources.Additionally, the content requested by the client computing devices mayhave a number of components, which can require further consideration oflatencies associated with delivery of the individual components as wellas the originally requested content as a whole.

With reference to an illustrative example, a requested Web page, ororiginal content, may be associated with a number of additionalresources, such as images or videos, that are to be displayed with theWeb page. In one specific embodiment, the additional resources of theWeb page are identified by a number of embedded resource identifiers,such as uniform resource locators (“URLs”). In turn, software on theclient computing devices, such as a browser application, typicallyprocesses embedded resource identifiers to generate requests for thecontent. Often the resource identifiers associated with the embeddedresource reference a computing device associated with the contentprovider such that the client computing device would transmit therequest for the additional resources to the referenced content providercomputing devices. Accordingly, in order to satisfy a content request,the content provider(s) (or any service provider on behalf of thecontent provider(s)) would provide client computing devices dataassociated with the Web page and/or the data associated with theembedded resources.

Some content providers attempt to facilitate the delivery of requestedcontent, such as Web pages and/or resources identified in Web pages,through the utilization and distribution of content at a number ofgeographically diverse data centers. Data centers typically maintain anumber of computing devices for maintaining content in a variety ofgeographically remote locations. By distributing content among more thanone data center, either by storing content at a particularly designatedlocation or by storing content at multiple locations, content providersseek to efficiently transmit requested content to client computingdevices, which are also typically located in a variety of geographiclocations.

Alternatively or additionally, some content providers attempt tofacilitate the delivery of requested content through the utilization ofa content delivery network (“CDN”) service provider. A CDN serviceprovider typically maintains a number of computing devices in acommunication network that can maintain content from various contentproviders. In turn, content providers can instruct, or otherwise suggestto, client computing devices to request some, or all, of the contentprovider's content from the CDN service provider's computing devices.

As with content providers, CDN service providers are also generallymotivated to provide requested content to client computing devices oftenwith consideration of efficient transmission of the requested content tothe client computing device and/or consideration of a cost associatedwith the transmission of the content. Accordingly, CDN service providersoften consider factors such as latency of delivery of requested contentin order to meet service level agreements or to generally improve thequality of delivery service.

Traditionally, a number of methodologies exist which measure theperformance associated with the exchange of data in any of theenvironments described above. For example, some methodologies providefor limited measurement of performance metrics associated with networkside processing of a content request. Other methodologies allow forlimited measurement of performance metrics associated with a contentrequest as measured from a browser application's perspective.

BRIEF DESCRIPTION OF THE DRAWINGS

Many of the attendant advantages and aspects of the present disclosurewill become more readily appreciated as the same become betterunderstood by reference to the following detailed description, whentaken in conjunction with the accompanying drawings, wherein:

FIG. 1 is a block diagram illustrative of a performance measurementsystem including a number of client computing devices, a contentprovider, a processing device, and a number of service providers;

FIG. 2 is a block diagram of the performance measurement system of FIG.1 illustrating the process of monitoring and fulfilling resourcerequests;

FIG. 3 is a block diagram of the performance measurement system of FIG.1 illustrating the process of identifying and providing performancemetric information from a client computing device;

FIG. 4 is a block diagram of the performance measurement system of FIG.1 illustrating the process of identifying and providing performancemetric information from a content provider;

FIG. 5 is a flowchart illustrative of a performance monitoring routineimplemented by a client computing device for monitoring the performanceassociated with resource requests made by the client computing device;

FIG. 6 is a flowchart illustrative of a performance monitoring routineimplemented by a performance measurement component for furthermonitoring client side performance associated with resource requestsmade by the client computing device;

FIGS. 7A and 7B are illustrative user interfaces displaying a variety ofperformance metric information collected by the performance measurementsystem of FIG. 1; and

FIG. 8 is a flowchart illustrative of a content processing andrecommendation routine implemented by the processing device of theperformance measurement system of FIG. 1 for processing resourcerequests made by the client computing device and determining whether touse a content delivery network service provider to service a particularresource request.

DETAILED DESCRIPTION

Generally described, the present disclosure is directed to monitoringthe performance and processing of data exchanges between clientcomputing devices and server computing devices. Specifically, aspects ofthe disclosure will be described with regard to monitoring a dataexchange involving a request by a client computing device for one ormore resources hosted, for example, by a content provider anddynamically identifying a number of service providers for alternativelyhosting, processing, and/or transmitting at least a portion of the oneor more resources. Data exchanges involving subsequent client requestsfor the one or more resources which are at least partially hosted,processed, and/or transmitted by an identified service provider can alsobe monitored. Performance data can then be used to assess performancerelated to the processing of the various client requests for the one ormore resources. Additionally, the processed performance data can be usedto determine whether to recommend one or more identified serviceproviders for improving performance of subsequent client requests forthe one or more resources. In yet further aspects, other types and/orcombinations of service providers can be identified and evaluated forimproving such performance.

Traditionally, network servers can collect latency informationassociated with a server's processing of a client request for aresource. For example, network servers can measure a time associatedwith processing an incoming client request, identifying/obtaining therequested resource, and initiating the transmission of the resourceresponsive to the client request. Additionally, client computing devicescan collect latency information associated with the client computingdevice's initiation of a resource request and receipt of the resourceresponsive to the request. Aspects of the present disclosure, which willbe described further below, are directed to identifying and providingadditional information to improve the performance assessment related tothe processing of a client request for one or more resources and todynamically identifying and evaluating modifications to the originalrequest, original resource, and/or any embedded resources. Althoughvarious aspects of the disclosure will be described with regard toillustrative examples and embodiments, one skilled in the art willappreciate that the disclosed embodiments and examples should not beconstrued as limiting.

FIG. 1 is a block diagram illustrative of a performance measurementsystem 100 for monitoring the performance and processing of dataexchanges. As illustrated in FIG. 1, the performance measurement system100 includes a number of client computing devices 102 (generallyreferred to as clients) for requesting content from a content provider.As illustrated in FIG. 1, each client computing device 102 includes aclient computing component 104 for requesting content from networkresources in the form of an originally requested resource that mayinclude identifiers to one or more embedded resources that need to berequested. As will be described in greater detail below, the clientcomputing component 104 also identifies performance metrics obtained byclient computing devices and/or components, such as browser softwareapplications. Additionally, the client computing device 102 includes aperformance measurement component 106 that identifies additionalperformance metrics associated with the client request, such as networklevel performance data including, for example, timing of receipt offirst and last network packets of data for fulfilling the originalresource request and each embedded resource request. In one embodiment,the performance measurement component 106 works in conjunction with theclient computing component 104 to collect performance metric informationsuch as from an operating system or a data file.

As illustrated in FIG. 1, the client computing component 104 andperformance measurement component 106 are executed on each clientcomputing device 102. Alternatively, the client computing component 104may not be configured, or is otherwise incapable of, obtaining orproviding some or all of the performance metric information describedherein. In such an embodiment, the client computing component 104 mayfunction with a reduced or limited capacity. In still a furtherembodiment, the client computing component 104 may function inconjunction with a separate communication software application (e.g., abrowser software application) to provide the combined functionalitydescribed for the client computing component 104. For example, theclient computing component could correspond to a stand alone softwareapplication, plugin, script, and the like. Additionally, although eachclient computing device 102 is illustrated as having a separateperformance measurement component 106, in an alternative embodiment, theperformance measure component 106 may be shared by one or more clientcomputing devices.

In an illustrative embodiment, the client computing devices 102 cancorrespond to a wide variety of computing devices including personalcomputing devices, laptop computing devices, hand-held computingdevices, terminal computing devices, mobile devices, wireless devices,various electronic devices and appliances and the like. As alsoillustrated in FIG. 1, the client computing devices 102 are consideredto be logically grouped, as represented generally by client 107,regardless of whether the client computing devices are physicallyseparate and geographically distributed throughout the communicationnetwork 114. In this regard, the client computing devices 102 can eachcommunicate directly or indirectly with other computing devices overnetwork 114, such as a wide area network or local network. Additionally,one skilled in the relevant art will appreciate that client 107 caninclude or otherwise be associated with various additional computingdevices/components including, but not limited to, content and resourceadministrative components, DNS resolvers, scheduling devices/components,and the like.

Each of the client computing devices 102 can accordingly includenecessary hardware and software components for establishingcommunications over the network 114. For example, the client computingdevices 102 may include networking components and additional softwareapplications that facilitate communications via the Internet or anintranet. As previously described, the client computing device 102 mayinclude an additional, separate browser software application. The clientcomputing devices 102 can also be associated with, or otherwise include,other computing components, such as proxy applications, for furtherfacilitating communications via the Internet or an intranet. Aspreviously described, the client computing components 104 can eachfunction as a browser software application for requesting content from anetwork resource. Additionally, in an illustrative embodiment, theperformance measurement component 106 of the client computing device 102can function as a proxy application for managing browser applicationcontent requests to the network resource. In other embodiments, theclient computing devices 102 may be otherwise associated with anexternal proxy application, as well as any other additional softwareapplications or software services, used in conjunction with requests forcontent.

With continued reference to FIG. 1 and as set forth generally above, theperformance measurement system 100 can include a content provider 108 incommunication with the one or more client computing devices 102 via thecommunication network 114. The content provider 108 can include a numberof content delivery components 110, such as a Web server component andassociated storage component corresponding to one or more servercomputing devices for obtaining and processing requests for content(such as Web pages) from the client computing devices 102, as well as anapplication server component for further processing various contentrequests. The content provider 108 can also include a performancemeasurement component 112 for measuring performance metrics, such as atime associated with processing an incoming client request,identifying/obtaining the requested resource, and initiating thetransmission of the resource responsive to the client request. Oneskilled in the relevant art will appreciate that the content provider108 can include or otherwise be associated with various additionalcomputing resources including, but not limited to, additional computingdevices for administration of content and resources, DNS name servers,interfaces for obtaining externally provided content (e.g.,advertisements, web services, etc.), and the like. Although theperformance measurement system 100 is illustrated in a client-serverconfiguration, one skilled in the relevant art will appreciate that theperformance measurement system 100 may be implemented in a peer-to-peerconfiguration as well.

With yet further continued reference to FIG. 1, the performancemeasurement system 100 can further include a processing device 116 forcollecting and aggregating performance data related to the processing ofclient requests. The processing device 116 can also be used to assessthe collected performance data and to determine if modifications to theoriginal resource and/or embedded resources should be made to improveperformance for subsequent client requests for the original resourceand/or embedded resources.

As illustrated in FIG. 1, the processing device 116 is in communicationwith the one or more client computing devices 102 and the contentprovider 108 via communication network 114. Additionally, as will befurther described below, the processing device 116 can include a metricprocessing component 118 for the collection and aggregation ofperformance data from the client computing devices 102 and contentprovider 108, as well as for the assessment of performance data.Specifically, in one embodiment, the client computing components 104 andperformance measurement components 106 associated with client computingdevices 102 provide performance metric information to the metricprocessing component 118, while the performance measurement component112 of the content provider 108 provides performance metric informationto the metric processing component 118. The processing device 116 canfurther include a local data store 120 for storing the receivedperformance data. It will be appreciated by one skilled in the art andothers that metric processing component 118 and data store 120 maycorrespond to multiple devices/components and/or may be distributed.

With continued reference to FIG. 1, the performance measurement system100 can further include a set 122 of one or more service providers, suchas CDN service provider 124, storage service provider 126, applicationservice provider 128, transit service provider 130, and a number ofother service providers 132. As also illustrated in FIG. 1, the set 122of one or more service providers is considered to be logically grouped,as represented generally by set 122, regardless of whether the serviceproviders are physically separate and geographically distributedthroughout the communication network 114. In this regard, each serviceprovider in the set 122 can communicate directly or indirectly withother computing devices, such as client computing devices 102 andcontent providers 108, over communication network 114, such as a widearea network or local network.

The CDN service provider 124 illustrated in FIG. 1 corresponds to alogical association of one or more computing devices (not shown)associated with a CDN service provider. Specifically, the CDN serviceprovider 124 can include a number of Point of Presence (“POP”) locationsthat correspond to nodes on the communication network 114. The CDN POPswill typically be geographically distributed throughout thecommunication network 114 in a manner to best serve various demographicsor other characteristics associated with client computing devices 102.Each CDN POP includes a DNS component made up of a number of DNS servercomputing devices for resolving DNS queries from the client computers102. Each CDN POP also includes a resource cache component made up of anumber of cache server computing devices for storing resources fromcontent providers and transmitting various requested resources tovarious client computers. The DNS components and the resource cachecomponents may further include additional software and/or hardwarecomponents that facilitate communications including, but not limited to,load balancing or load sharing software/hardware components.

With continued reference to FIG. 1, the storage service provider 126also corresponds to a logical association of one or more computingdevices (not shown) associated with a storage provider. As with the CDNservice provider 124, the storage service provider 126 can also includea number of Points of Presence (“POP”) locations that correspond tonodes on the communication network 114. In one illustrative embodiment,these storage POPs correspond to data centers which can be used toprovide storage system facilities and maintain resources on behalf of acontent provider. Accordingly, each storage POP, or data center, caninclude a number of components for, amongst other things, storing andproviding resources on behalf of a content provider. Such resources caninclude text, images, sounds, videos, animations, as well as userprofile information, applications, web services, and the like. Inaddition, the storage POPs, or data centers, will typically begeographically distributed throughout the communication network 114 in amanner to best serve various demographics of client computing devices102.

The application service provider 128 illustrated in FIG. 1 alsocorresponds to a logical association of one or more computing devices(not shown) associated with an application service provider. As with theCDN and storage service providers 124 and 126, the application serviceprovider 128 can also include a number of Points of Presence (“POP”)locations that correspond to nodes on the communication network 114.Each application POP includes a network computing (or cloud computing)component (NCC) for hosting applications, such as data streamingapplications, via a number of instances of a virtual machine, generallyreferred to as an instance of an NCC. One skilled in the relevant artwill appreciate that NCCs would include physical computing deviceresources and software to provide the multiple instances of a virtualmachine or to dynamically cause the creation of instances of a virtualmachine. Each application POP may also include a storage component madeup of a number of storage devices for storing resources from contentproviders which will be processed by an instance of an NCC andtransmitted to various client computers. In addition, the applicationPOPs will typically be geographically distributed throughout thecommunication network 114 in a manner to best serve various demographicsof client computing devices 102.

With continued reference to FIG. 1, the transit service provider 130also corresponds to a logical association of one or more computingdevices (not shown) associated with a transit service provider. Atransit service provider provides computing devices and/or services,such as routers, bridges, gateways, network access services, and thelike, for establishing network communications over or between networks.In one illustrative embodiment, the transit service provider 130 may bean Internet service provider (ISP). As with the foregoing serviceproviders, the transit service provider 130 can also include a number ofPoints of Presence (“POP”) locations that correspond to nodes on thecommunication network 114. In addition, the transit POPs will typicallybe geographically distributed throughout the communication network 114in a manner to best serve various demographics of client computingdevices 102.

In further reference to FIG. 1, the performance measurement system 100can include a number of other service providers 132 for use in hosting,processing, or enabling the transmission of content. Moreover, it willbe appreciated by one skilled in the relevant art that a serviceprovider may offer any one or more of the services described above asbeing associated with CDN, storage, application, and transit serviceproviders. For example, a data center may provide both traditionalstorage services, as well as network computing services.

As will be described further below, the performance measurement system100 may use and evaluate performance of any one or more of the serviceproviders 124, 126, 128, 130, 132 in hosting and/or processing contenton behalf of the content provider 108 and/or enabling the transmissionof such content. In one illustrative embodiment, the performancemeasurement system 100 may evaluate the performance associated withproviding a Web page to a client computing device 102 using a particularCDN service provider to host and/or process the associated content forthe Web page. In another illustrative embodiment, the performancemeasurement system 100 may evaluate the performance of each of a numberof data centers for storing and providing content for a particular Webpage. In yet another illustrative embodiment, the performancemeasurement system 100 may evaluate the performance associated with useof a particular data center connected using a particular transitprovider to provide content to a client computing device on behalf of acontent provider.

One skilled in the relevant art will also appreciate that the componentsand configurations provided in FIG. 1 are illustrative in nature.Accordingly, additional or alternative components and/or configurations,especially regarding additional components, systems and subsystems forfacilitating communications may be utilized.

With reference now to FIGS. 2-4, an illustrative example of theoperation of the performance measurement system 100 according to someembodiments will be described. For purposes of the example, however, theillustration has been simplified such that many of the componentsutilized to facilitate communications are not shown. One skilled in therelevant art will appreciate that such components can be utilized andthat additional interactions would accordingly occur without departingfrom the spirit and scope of the present disclosure.

With reference to FIG. 2, a client computing component 104 initiates acontent request that is intended to ultimately be received and processedby the content provider 108. In an illustrative embodiment, therequested content can correspond to a Web page that is displayed on theclient computing device 102 via the processing of a base set ofinformation, such as hypertext markup language (“HTML”), extensiblemarkup language (“XML”), and the like. The base set of information canalso include a number of embedded resource identifiers that correspondsto resource objects that should be obtained by the client computingdevice 102 as part of the processing of the requested content. Theembedded resource identifiers can be generally referred to as resourceidentifiers or resource URLs. The request for the base set ofinformation and the subsequent request(s) for any embedded resources canbe referred to generally as a “resource request.”

In one embodiment, prior to initiating a resource request, the clientcomputing component 104 associates a record identifier with the resourcerequest. As will be described further below, the record identifier canbe used to track performance metrics associated with processing therequested resource and any embedded resources. In one example, therecord identifier may be attached to the resource request as a header orotherwise embedded in the request. The client computing component 104then transmits the resource request with the record identifier. However,as will also be described further below, the client computing component104 can alternatively transmit the associated record identifier in aseparate transmission from the resource request.

It will be appreciated by one skilled in the relevant art and othersthat the client computing component 104 can generate the resourcerequest and associated record identifier itself or receive one or theother or both from another storage or computing device. For example,another computing device, such as processing device 116, can be used todetermine whether a test to monitor performance metrics associated withprocessing a particular resource, such as a Web page, should beconducted. In this example, the processing device 116 can send the testrequest, which includes a resource identifier corresponding to thedesired resource request and a record identifier further associated withthe resource identifier, to the client computing device 102.

In one illustrative embodiment, as shown in FIG. 2, the client computingcomponent 104 initiates the content request by transmitting the resourceidentifier and associated record identifier directly or indirectly tothe performance measurement component 106 of the client computing device102. However, it will be appreciated by one skilled in the relevant artthat, in the alternative, the performance measurement component 106 canotherwise intercept the content request initiated by the clientcomputing component 104.

Continuing with the present example and in further reference to FIG. 2,the performance measurement component 106 receives the resource requestand forwards the resource request on to the content provider 108 viacommunication network 114. Thereafter, the performance measurementcomponent 106 continually monitors performance metrics associated withthe processing of the requested resource, including any embeddedresources. Specifically, in one illustrative embodiment, the performancemeasurement component 106 monitors network level performance metricsassociated with the processing of the requested resource and anyembedded resources, such as timing of receipt of the first and lastbytes (or packets) of data of each request. The performance measurementcomponent 100 can either obtain such performance metric informationdirectly from the operating system of the client computing device 102 orthrough the client computing component 104. The performance measurementcomponent 106 associates the monitored performance metrics with therecord identifier.

As further illustrated in FIG. 2, the content provider 108 receives theresource request from the client computing device 102 and processes theresource request using content delivery components 110, such as a Webserver. The content provider 108 can also use a performance measurementcomponent 112 to monitor performance metrics associated with processingthe incoming client request, identifying/obtaining the requestedresource, and initiating the transmission of the resource responsive tothe client request. As shown in FIG. 2, upon obtaining the requestedresource, the content provider 108 initiates transmission of therequested resource to the client computing device 102.

In this illustrative example, the performance measurement component 106at the client computing device 102 obtains the requested resource,continues monitoring the processing of the requested resource, andforwards the requested resource to the client computing component 104.For example, the performance measurement component 106 may serve as aproxy application for receiving the requested resource or otherwiseintercepting the requested resource. The client computing component 104also tracks performance metrics associated with the processing of therequested resource. Upon receipt of the requested resource, the clientcomputing component 104 begins processing the content. For example, theclient computing component 104 can process the content for display on amonitor or other display device associated with the client computingdevice 102. Alternatively, the client computing component 104 canprocess the content for sending to any other component or externaldevice (e.g., a framebuffer). As will be further described below, theabove described functions apply to the processing of the originallyrequested resource, as well as any embedded resources.

With reference now to FIG. 3, the client computing component 104 and theperformance measurement component 106 of the client computing device 102can each identify performance metric information that the respectivecomponents have monitored and/or collected. The performance metricinformation from the client computing component 104 can include avariety of information, such as process information, memory information,network data, resource data, client computing component information,including page setups, browser rendering information, state variables,and other types of information. In one specific example, the performancemetric information can include information regarding a time at which aparticular resource was rendered on a Web page, its location on thepage, whether the resource was rendered on the device display, and thelike. The performance metric information from the performancemeasurement component 106 of the client computing device 102 can alsoinclude a variety of information as similarly set forth generally above.In one specific example, the performance metric data can include networkstatistics, latencies, bandwidths, and data arrival times, such as thetiming of receipt of first and last packets of information for therequested resource and each embedded resource. In another specificexample, the performance metric information can include timinginformation associated with processing executable resources, such asJavaScript, as well as additional information that can be used toindirectly determine processing times associated with the execution ofthe resource once the executable code has been obtained.

The performance metric information from the client computing component104 and/or the performance measurement component 106 of the clientcomputing device 102 can also include basic resource information, suchas an identification of the resource type, a link to a header associatedwith the requested resource, a size of the resource, and the like. Evenfurther, the performance metric information can include underlyingcomputer resource information, such as a resolution of the display ofthe client computing device 102, a version of the browser applicationsoftware, an identification of any plugins associated with the browserapplication software, an identification of any updates to the operatingsystem of the client computing device 102, and the like. Even further,the performance metric information can include information regarding thelocation of the client device 102 (such as an IP address), serversassociated with the content provider 108, and/or servers associated witha service provider 122 (e.g., for a CDN service provider 124, a CDNservice provider DNS nameserver or assigned cache server).

With continued reference to FIG. 3, the client computing component 104and the performance measurement component 106 of the client computingdevice 102 provide the identified performance metric informationtogether with the associated record identifier of the requested resourceto the metric processing component 118 of the processing device 116 viathe communication network 114. The metric processing component 118 thenprocesses the received performance metric information to assessperformance related to the processing of the client request for theoriginal resource and any embedded resources. The processed performancemetric information can be used to support modifications to the originalresource and/or embedded resources to improve performance for subsequentclient requests for the original resource. As will be appreciated by oneskilled in the art and others, the processing device 116 can store thereceived and/or processed performance metric information in local datastore 120, or any other data store distributed across the network 114.Additionally, as will be further described below in reference to FIGS.7A and 7B, the processing device 116 can cause the display of theprocessed performance metric information to a user of the system forfurther assessment.

In one illustrative embodiment, once the client computing component 104completes processing of the requested resource and any embeddedresources, the client computing component 104 identifies performancemetric information that the client computing component 104 monitoredand/or otherwise collected related to such processing. In this example,the client computing component 104 provides the identified performancemetric information with the record identifier associated with therequested resource to the metric processing component 118. Upon receiptof this information, the metric processing component 118 then requestsany further performance metric information related to the requestedresource and any embedded resources from the performance measurementcomponent 106 of the client computing device 102. In response, theperformance measurement component 106 of the client computing device 102identifies and provides performance metric information with the recordidentifier associated with the requested resource to the metricprocessing component 118. The metric processing component 118 can usethe record identifier to aggregate the received performance metricinformation. It will be appreciated by one skilled in the art and othersthat the identified performance metric information can be transmitted tothe metric processing component 118 by a number of alternativemethodologies and/or components.

In another illustrative embodiment, and as will be described furtherbelow in reference to FIG. 8, the metric processing component 118 canuse the processed performance metric information associated with theoriginal resource and any embedded resources to dynamically determine anadditional or alternative service provider, such as CDN service provider124, storage service provider 126, application service provider 128, orthe like, for hosting or processing at least a portion of the originalresource and/or any embedded resources to improve performance. As willalso be further described below, in making such a determination, themetric processing component 118 can further take into considerationperformance metric information collected and associated with subsequentresource requests for the original resource and any embedded resourcesusing such alternative service providers, as well as service providerselection criteria which can be obtained from the original contentprovider.

With reference now to FIG. 4, in one illustrative embodiment, theperformance measurement component 112 of the content provider 108 canidentify performance metric information that it has collected related tothe processing of the requested resource and/or any embedded resource.The performance measurement component 112 provides the identifiedperformance metric information to the metric processing component 118 ofthe processing device 116 via communication network 114. As will beappreciated by one skilled in the art and others, the performancemeasurement component 112 of the content provider 108 can provide theperformance metric information upon request from the processing device116 or upon completing its processing of the requested resource. As willbe described further below, the processing device 116 can then aggregatethe performance metric information from all components for displaying,processing, storing, or otherwise assessing performance related to theprocessing of the requested resource.

With reference now to FIG. 5, one embodiment of a performance monitoringroutine 500 implemented by the client computing component 104 of theclient computing device 102 will be described. One skilled in therelevant art will appreciate that actions/steps outlined for routine 500may be implemented by one or many computing devices/components that areassociated with the client computing device 102. Accordingly, routine500 has been logically associated as being generally performed by theclient computing device 102, and thus the following illustrativeembodiments should not be construed as limiting.

At block 502, a client computing component 104 identifies an originalresource request. As previously mentioned, the client computingcomponent 104 can generate the original resource request or receive theoriginal resource request from another computing device, such asprocessing device 116. In one example, the original resource request canbe for a Web page, such as http://example.com. At block 504, the clientcomputing component 104 associates a record identifier (RID) with theoriginal resource request. The RID may be a unique identifier associatedwith the original resource request. As will be further described below,the RID can also be associated with any embedded resources included in aresponse to the original resource request. Even further, although notillustrated, in an alternative embodiment, in the event that the clientcomputing component 104 does not need a RID, the client computingcomponent 104 may not associate a RID with the resource request at shownat block 504.

At block 506, the resource request is transmitted to another entity. Inthis example, the resource request is transmitted to the performancemeasurement component 106 of the client computing device 102. Aspreviously mentioned, the performance measurement component 106 canalternatively intercept the transmission request as it is being routedto a content provider 108 for example. In one illustrative embodiment,the resource request may itself contain the RID, such that the resourcerequest and associated RID are transmitted as part of the sametransmission. For example, the RID can be included as a portion of theresource URL used to request the resource. Alternatively oradditionally, the RID may be transmitted in a second communication,either before or after the transmission including the resource request.For example, a “start new request group” command, including the RID maybe issued before or after the initial resource request. In one furtheralternative embodiment, the client computing component 104 may notinclude a RID with the issuance of a “start new request group” command,and in this case, the performance measurement component 106 maygenerate, or otherwise obtain, such a RID upon receipt of the “start newrequest group” command.

Continuing at block 508, a determination is made at the client computingcomponent 104 regarding whether any additional resources need to berequested to fulfill the original resource request. As appreciated byone skilled in the relevant art, a response to the original resourcerequest can be returned to the client computing component 104 whichincludes a number of resource URLs corresponding to a number of embeddedresources required to fulfill the original resource request. In oneembodiment, if such additional resources are identified, processingreturns to block 506 where the client computing component 104 transmitsone or more requests for the identified embedded resources with the RIDassociated with the original resource request.

Alternatively or additionally, the client computing component 104 canassign a component record identifier (CRID) to each request for anembedded resource at optional block 510. In this example, whenprocessing returns to block 506, the client computing component 104 cantransmit the one or more embedded resource requests with therespectively assigned CRIDs. In an illustrative embodiment, the requestsfor embedded resources can be transmitted with respective CRIDs alone ortogether with the RID of the original resource request. As embeddedresource requests (or component requests) are fulfilled, the returnedcontent is processed by the client computing component 104. It will beappreciated by those skilled in the art and others that a response to anembedded resource request can include links to further embeddedresources. As such, the functionality associated with blocks 506-510 maybe repeated as described above until no resource requests areoutstanding and no more additional resources need to be requested.

It will be appreciated by one skilled in the relevant art that resourcerequests are processed by the client computing device 102 in accordancewith logic associated with the particular configuration of the browsersoftware application. For example, the browser software application maybe limited by a number of resource requests that can be made at onetime, an order associated with the type of requests that can be made, anorder based on a predetermined location for the requested resources on adisplay screen, or other limitations provided in the requested baseresource.

Once the client computing component 104 determines at block 508 that noadditional resources need to be obtained to fulfill the originalresource request or any subsequent embedded resource requests,processing can continue at optional block 512. At block 512, atermination command, such as “end new request group”, may be transmittedto indicate that the request, including requests for all embeddedresources, has completed. Such a termination command may provide closureto a “start new request group” command, if one were issued as part ofthe first iteration of block 506. In this example, the start/terminationcommands can be received and used by the performance measurementcomponent 106 to determine which requested resources are associated witha particular originally requested resource.

At block 514, once the client computing component 104 has completedprocessing the requested original resource and any embedded resources,the client computing component 104 provides monitored performance metricinformation to processing device 116. The client computing component 104monitors such performance metric information throughout the processingof the original resource request from initiation of the originalresource request to final rendering of the requested resource and anyembedded resources. The performance metric information can include, forexample, timing data associated with the initiation of each request,receipt of a response to each request, and rendering of each requestedresource, as well as other information as described herein. The routine500 ends at block 516.

With reference now to FIG. 6, one embodiment of a performance monitoringroutine 600 implemented by the performance measurement component 106 ofthe client computing device 102 will be described. One skilled in therelevant art will appreciate that actions/steps outlined for routine 600may be implemented by one or many computing devices/components that areassociated with the client computing device 102. Accordingly, routine600 has been logically associated as being generally performed by theclient computing device 102, and thus the following illustrativeembodiments should not be construed as limiting.

At block 602, the performance measurement component 106 of the clientcomputing component 100 receives (or intercepts) an original resourcerequest from the client computing component 104. In one illustrativeembodiment, the performance measurement component 106 receives the RIDwith the original resource request. Alternatively, the RID can beprovided as a part of a separate transmission, and accordingly, in thiscase, the performance measurement component 106 receives the RIDseparately. At block 604, the performance measurement component 106associates the RID with the original resource request. In accordancewith other embodiments discussed above, the original resource requestmay be preceded or followed by a command or instructions, such as a“start new request group” command. Such commands may be transmitted withor without a RID, as set forth above. If such commands are received atthe performance measurement component 106 without a RID, the performancemeasurement component can generate, or otherwise obtain, a RID toassociate the original resource request at block 604.

Continuing at block 606, the original resource may be requested, such asby proxying or forwarding the resource request to the content provider108 via network 114. The resource request may be modified from itsoriginal form before sending, such as by stripping headers including theassociated RID. The performance measurement component 106 also monitorsthe processing, including fulfillment, of the resource request at block606. For example, the performance measurement component can identifyperformance metric information related to the initiation of the resourcerequest, the receipt of first and last bytes of data for each requestedresource and any embedded resources, the receipt of responsive content,and the like. As will be appreciated by one skilled in the relevant art,once a response to the resource request is received at the performancemeasurement component 106, the response is returned to the requestingapplication.

At block 608, a determination is made by the performance measurementcomponent 106 regarding whether a subsequent resource request related tothe original resource request has been made by the client computingcomponent 104 and accordingly received (or intercepted) by theperformance measurement component. If a subsequent embedded resourcerequest (which may bear the same RID as the original resource request,an appropriate CRID, and/or be within a start/stop command window) isreceived, processing continues at block 610. At block 610, theperformance measurement component 106 requests any embedded resourcesand monitors the processing of the requested embedded resources assimilarly described above in reference to the originally requestedresource and block 606. The functionality associated with blocks 608-610may be repeated as described above until no resource requests areoutstanding.

If the performance measurement component 106 determines that no moreoutstanding resource requests remain at block 608, processing continuesat block 612. Specifically, the performance measurement component 106provides monitored performance metric information to processing device116. The performance measurement component 106 monitors such performancemetric information throughout the processing of the original resourcerequest, from initiation of the original resource request to finalrendering of the requested resource and any embedded resources. Theperformance metric information can include, for example, timing dataassociated with the initiation of each request, receipt of a response toeach request, and receipt of first and last packets of data for each ofthe original resource request and any embedded resource requests, aswell as other additional information as described herein.

In one illustrative embodiment, the performance measurement component106 can identify performance metric information for providing to theprocessing device 116 in a variety of ways. For example, in oneembodiment, the performance measurement component 106 can storeperformance measurement information in a log file together withidentifiers to associate performance metric information withcorresponding resource requests. In this example a set of requestedresources may be joined by common RIDs, common CRIDs, associated CRID(e.g., where each component has a distinct CRID, but the distinct CRIDsof a single group have been associated or otherwise linked together,such as by a RID). In another illustrative embodiment, the performancemeasurement component can retrieve performance metric information from alog file based on timing information associated with a resource request.For example, a set of requested resources may be defined as theresources requested or fulfilled between a start command and an endcommand, or between an original resource request (inclusive) and a stopcommand. The routine 600 ends at block 614.

With reference now to FIG. 7A, an illustrative user interface 700generated by the processing device 116 for displaying a variety ofperformance metric information collected, or otherwise identified, bythe performance measurement system 100 of FIG. 1 will be described.Generally, the user interface 700 shown in FIG. 7A provides a graphicalside-by-side comparison of the performance metric information identifiedfor the originally requested resource and some or all requested embeddedresources. The user interface 700 may also be provided over the network114 for display on other computing devices.

With reference to FIG. 7A, the user interface 700 may be utilized todisplay a set of time-based events for a set of resources. For example,the user interface 700 may graphically represent an order of time-basedevents for an originally requested resource and for each subsequentrequest for embedded resources. More specifically, the user interface700 includes a legend 702 identifying, for a number of resource types, agraphical indicator corresponding to a number of time-based events 704,706, 708, 710, and 712 involved in processing a request for theresource. The resource types identified in the legend 702 include HTMLresources, image (IMG) resources, and JavaScript (JS) resources.However, it will be appreciated that a number of alternative oradditional resource types can be identified. For each resource type, thelegend 702 provides a distinct color-coded indicator corresponding to atransition period and/or transition event(s) occurring between eachidentified event 704, 706, 708, 710, and 712. In one embodiment, thedistinct indicators may be visual in nature, such as color-coded,cross-hatched, or the like. In another embodiment, instead of using adistinct indicator for each transition period and/or transition event(s)associated with each resource type as illustrated in FIG. 7A, a distinctindicator may be used simply for each transition period and/ortransition event(s) regardless of the resource type.

In an illustrative embodiment, events 704, 706, 708, 710, and 712correspond to the following time-based events identified by theperformance metric information. Event 704 identifies a Start Eventrepresenting a time at which the corresponding resource was known to berequired by the client computing component 104. Event 706 identifies aNetStart Event representing a time at which the corresponding resourcewas actually requested by the client computing component 104. The timingof the NetStart Event may not be the same as the Start Event if, forexample, the browser software application limits the number ofconcurrent connections with a particular domain. Event 708 identifies aFirst Byte Event representing a time at which the first byte (or firstpacket) of the requested resource is received by the performancemeasurement component 106 of the client computing device 102. Event 710identifies a Last Byte Event representing a time at which the last byte(or last packet) of the requested resource is received by theperformance measurement component 106 of the client computing device102. Finally, event 712 identifies a Render Event representing a time atwhich the client computing component 104 finishes rendering therequested resource.

A second portion 730 of the user interface 700 corresponds to arepresentation illustrating the occurrence of each of the time-basedevents 704, 706, 708, 710, and 712 for all or some of the resourcesrequested in resolving the original resource request. In one embodiment,the representation horizontally corresponds to time and verticallycorresponds to an ordered listing of the requested resources. In oneexample, the order can specifically correspond to an order in which therequested resources are initially identified by the client computingcomponent 104. In addition, the second portion 730 of the displayincludes a variety of additional information adjacent to the time-basedevent representation for each resource. For example, in a first column732, a resource type for each resource may be provided, e.g., HTML,image, CSS, JavaScript, and the like. In a second column 734, a link toa header corresponding to each requested resource may be provided. In athird column 736, an HTTP response status code corresponding to eachrequested resource can be provided. Code 200, for example, is indicativeof a standard response for successful HTTP requests. Finally, in afourth column 738, the size of each resource can be provided.

In another embodiment, yet further additional information may bedisplayed in the user interface 700. For example, the user interface 700may display the total processing time, both numerically and graphically,associated with processing the original resource request including anyembedded resource requests. In this example, an indicator 740 mayillustrate a starting time while an indicator 746 may illustrate anending time, both associated with the processing of the originalresource request as a whole. Additionally, when the original resourcerequest is a request for a Web page, the user interface 700 mayillustrate a time, both numerically and graphically, at which allresources have been rendered in a portion of a Web page which isinitially visible to a user without scrolling. This portion of the Webpage is often referred as an “above the fold,” “above the scroll,” or“above the crease” portion. An indicator 744 in the user interface 700of FIG. 7A illustrates an “above the fold” (ATF) event.

The foregoing performance metric information provided in the userinterface 700 may be identified and/or collected by a combination of theclient computing component 104 and/or the performance measurementcomponent 106 of the client computing device 102. However, it will beappreciated by those skilled in the art and others that additionalperformance metric information can be displayed. Such additionallydisplayed performance metric information can be obtained by the clientcomputing device 102, by the performance measurement component 112 ofthe content provider 108, or based on further processing of any of theidentified and/or collected performance metric information. It will alsobe appreciated by one skilled in the relevant art that each resourceand/or each type of resource may be associated with all or only aportion of the above-described events and/or performance metricinformation. In addition, other events and/or indicators associated withthe other events may be used and illustrated in the user interface 700.

In one specific example, an executable resource, such as a JavaScriptresource, is not rendered and, accordingly, neither a Render Event 712nor an associated indicator illustrating the transition between a LastByte Event 710 and a Render Event 712 will be illustrated in the userinterface 700 for that executable resource. However, the processingdevice 116 can indirectly determine and display a processing timeassociated with execution of the code once the code itself is obtained(i.e., receipt of the last byte of the code which corresponds to theLast Byte Event 710). Such processing time is inferred in the userinterface 700 of FIG. 7A by illustration of a gap formed between thereceipt of the last byte of code associated with a first JavaScriptresource at 750 and the start event associated with a subsequentlyrequested JavaScript resource at 752. Alternatively, an additional eventand/or associated indicator could be used to specifically identify theprocessing time associated with execution of the code.

By providing and displaying the foregoing performance metric informationas set forth above, a user of the processing device 116 can readilyevaluate the performance associated with processing the originallyrequested resource, including any embedded resources. In particular, theuser interface 700 can help a user identify any problems associated withthe processing of the originally requested resource, as well asdetermine one or more solutions to the identified problem. Solutions forimproving performance may include, for example, making changes to thecontent itself, to the organization of content within the originallyrequested resource, to the client computing component, and the like.

Additionally, the user interface 700 can be used to illustrate arecommendation associated with the processed and displayed performancemetric information. For example, and as will be described further below,the processing device 116 may dynamically identify a number of serviceproviders 122, such as CDN service provider 124, storage serviceprovider 126, application service provider 128, and the like, foradditionally or alternatively hosting or processing at least a portionof the one or more resources associated with the original base resourcerequest and initiate testing of a subsequent request for the sameresources. As similarly set forth above with respect to the originalbase resource request, the user interface 700 can be used to displayperformance metric information associated with the processing of each ofthese subsequent requests. In addition, the user interface 700 can beused to display a recommendation identifying a particular serviceprovider which, for example, has been tested and demonstrated improvedperformance associated with processing the requested resources.

In one embodiment, FIG. 7A illustrates the performance associated withprocessing a request for an original resource and a number of embeddedresources from a content provider which maintains and provides both theoriginal and embedded resources, for example. Alternatively, FIG. 7Billustrates the performance associated with processing a subsequentrequest for the original and embedded resources from a content providerwhich provides the original resource but additionally or alternativelyuses a set 122 of one or more service providers to host or process oneor more of the embedded resources, for example. As illustrated by acomparison of the performance information illustrated in FIGS. 7A and7B, the use of the set 122 of one or more service providers to host orprocess the one or more embedded resources improved performanceassociated with processing a request for the original resource and theembedded resources as demonstrated by the overall reduced processingtime associated therewith. The user interfaces illustrated in FIGS. 7Aand 7B can also be provided to the content provider along with aspecific recommendation, for example, to consider using the set 122 ofone or more service providers to host or process the one or moreembedded resources in order to improve performance.

With reference now to FIG. 8, one embodiment of a content processing andrecommendation routine 800 implemented by the processing device 116 ofthe performance measurement system 100 will be described. One skilled inthe relevant art will appreciate that actions/steps outlined for routine800 may be implemented by one or many computing devices/components thatare associated with the processing device 116. Accordingly, routine 800has been logically associated as being generally performed by theprocessing device 116, and thus the following illustrative embodimentsshould not be construed as limiting.

At block 802, the processing device 116 identifies a set 122 of serviceproviders to be utilized to process a resource request for an originalresource and any embedded resources. An identified set of serviceproviders may include one or more service providers utilized inconjunction with various aspects related to the processing of theresource request(s), including one or more of the same type of serviceprovider or any combination of different service providers, such as CDNservice provider 124, storage service provider 126, application serviceprovider 128, transit service provider 130, and the like. The identifiedset of service providers may be used to assess performance associatedwith hosting, processing, and/or enabling transmission of at least aportion of the content associated with an original resource and anyembedded resources. Illustratively, a service provider can be used tohost just the original base resource, e.g., the HTML code, one or moreembedded resources, or any combination thereof. In addition, theprocessing device 116 can take into consideration a variety ofinformation for identifying a set of service providers.

For example, in one embodiment, the processing device 116 can receive arequest from a content provider to test specifically identified serviceproviders in order to assess performance associated with the hosting,processing, and/or enabling transmission of at least a portion of thecontent associated with an original resource and any embedded resources.In another embodiment, the processing device 116 can dynamicallyidentify, based on previously processed performance metric informationassociated with a request for an original resource and any embeddedresources provided by a content provider, one or more service providersthat could be used to host one or more of the resources to offerimproved performance. Alternatively, in yet another embodiment, theprocessing device 116 can automatically decide to test, and henceidentify, a number of service providers regardless of the assessedperformance associated with processing the original resource and anyembedded resources provided by the content provider without using aservice provider. For combinations of service providers, the processingdevice 116 can take into account additional business logic/relationshipsin pairing service providers.

In addition or alternatively, the processing device 116 can take intoconsideration a variety of service provider selection criteria. Theservice provider selection criteria can include, for example, serviceprovider quality of service information, cost information associatedwith processing a resource request using a particular service provider,and the like. The quality of service information can include informationregarding reliability, service level quality, transmission errors, andthe like. Specifically, in one embodiment, the processing device 116 canobtain service provider selection criteria from the content provider 108where the requested content originates. The content provider 108 maywant the processing device 116 to only test service providers which meeta minimum quality of service level or which would only cost a specifiedamount to implement.

At block 804, once the processing device 116 identifies a set of serviceproviders to use in processing a resource request, the processing device116 enables one or more resources corresponding to the resource requestto be hosted, processed, and/or transmitted by each identified serviceprovider. Specifically, in one embodiment, the processing device 116determines, for each identified service provider, configurationinformation for enabling the use of the service provider to host,process, and/or transmit one or more resources originating from thecontent provider 108. The processing device 116 uses the configurationinformation to prepare the service provider for hosting, processing,and/or transmitting content as necessary.

In one illustrative embodiment, where the resource request correspondsto a request for a Web page, the processing device 116 can continue touse the original resource identifier for the original resource request.In this embodiment, the content provider 108 continues to maintain andprovide the HTML code that is responsive to the original resourcerequest. However, in one example, an identified CDN service provider 124can be prepared to provide various resources embedded in the HTML codereturned by the content provider 108. Additionally, an identifiedapplication service provider 128 can be prepared to host an applicationcorresponding to other resources in the base HTML code. Accordingly, theprocessing device 116 can modify at least a portion of the HTML codewith translated embedded resource requests. Specifically, one or moreembedded resource identifiers can be modified such that they are nowdirected to the CDN service provider 124 or the application serviceprovider 128, for example. Instead of modifying the HTML code, theperformance measurement component 106 of the client computing device 102can dynamically translate requests for one or more embedded resourcessuch that they are directed to a designated service provider, such asthe CDN service provider 124 or the application service provider 128.Illustratively, the performance measurement component 106 would receivea client request corresponding to a resource as identified in theoriginal HTML and dynamically translate the resource request as if theoriginal HTML had been modified. Accordingly, the resource request wouldthen be directed to the desired service provider. In other embodiments,other computing components, such as DNS servers or IP routingcomponents, can alternatively be used to intercept and/or translateresource requests in order to test performance associated with otherdesignated service providers.

In another aspect, the identified set of service providers may be one ormore storage service providers, such as one or more geographicallydistributed data centers for hosting one or more resources on behalf ofthe content provider 108. In this embodiment, the one or more storageservice providers 126 can be prepared to provide the resources embeddedin the HTML code returned by the content provider 108. Accordingly, theprocessing device 116 can modify at least a portion of the HTML codewith translated embedded resource requests. Specifically, one or moreembedded resource identifiers can be modified such that they are nowdirected to the one or more storage service providers 126.Alternatively, as similarly set forth above, computing components, suchas the performance measurement component 106, DNS servers, IP routingcomponents, and the like, can be used to intercept and/or translate theresource requests to test performance associated with using the one ormore storage service providers.

In another embodiment, the processing device 116 can store the HTML codeof the Web page to be tested on a local server. In this case, theprocessing device 116 re-writes the original resource identifier toquery the processing device 116 (or associated Web server) for therequested resources. For example, the processing device 116 can modifythe original resource identifier ashttp://www.processingdevice.com/contentprovider.com/path/resource.xxx.In this embodiment, the processing device 116 would provide the modifiedHTML that would include translated embedded resource identifiers.

Returning to FIG. 8, at block 806, the processing device 116 theninitiates the resource request associated with content hosted and/or tobe processed and/or transmitted by the set of service providers byrequesting that the client computing device 102 initiate the query. Assimilarly described above, the client computing device 102 monitors andcollects performance data associated with the processing of the resourcerequest and provides the performance data to the processing device 116.Accordingly, at block 810, the processing device 116 obtains andprocesses the performance data from the client computing device 102. Theobtained performance data is associated with the processing of theresource request using the set of service providers to provide one ormore of the resources corresponding to the resource request.

Next, at block 812, a determination is made whether any additional setsof service providers should be used to host one or more of the resourcescorresponding to the resource request and, accordingly, be tested todetermine how the use of the additional sets of service providers mayaffect the performance associated with processing such a request. If anadditional set of service providers is identified, then processingreturns to block 804 and the foregoing process in reference to blocks804-812 is repeated as described above. If no additional set of serviceproviders are identified, processing continues at block 814.

At block 814, the processing device 116 dynamically determines a set ofservice providers based on the obtained and processed performance data.In an illustrative embodiment, the processing device 116 can alsoconsider service provider selection criteria in the determination of aset of service providers. As similarly mentioned above, the serviceprovider selection criteria can be obtained from a content provider 108and can include quality of service information, cost information,service provider location preference information, and the like. As alsoset forth above, the quality of service information can includeinformation regarding reliability, service level quality, transmissionerrors, and the like. In one example, the processing device 116 candetermine that a set of service providers corresponding to the bestperformance data is the determined set of service providers.Alternatively, a content provider 108 may not want to implement the bestperforming set of service providers for hosting, processing, and/ortransmitting one or more resources in an associated Web page, but ratherwants to take into consideration cost. Accordingly, in that case, theprocessing device 116 may select the set of service providers associatedwith the best performing set of service providers within a particularcost range.

In addition to determining the set of service providers, the processingdevice 116 can also generate a recommendation identifying the determinedset of service providers or provide an evaluation of all sets of serviceproviders together with a recommendation of the determined set ofservice providers. Such recommendations and/or evaluations can then beprovided to the content provider 108. The processing device 116 can alsogenerate and provide re-written HTML code with translated embeddedresources to the content provider 108 for utilizing the determined setof service providers. The routine ends at block 816.

It will be appreciated by those skilled in the art and others that whileprocessing, monitoring, and other functions have been described hereinas being performed at various components of the client computing device102 and/or the processing device 116, these functions can be distributedacross one or more computing devices. In addition, the performancemetric information monitored at the client computing device 102 can bemaintained globally by the client computing device 102 and shared withall or some subset of the components of the client computing device 102.

It will further be appreciated by those skilled in the art and othersthat all of the functions described in this disclosure may be embodiedin software executed by one or more processors of the disclosedcomponents. The software may be persistently stored in any type ofnon-volatile storage.

Conditional language, such as, among others, “can,” “could,” “might,” or“may,” unless specifically stated otherwise, or otherwise understoodwithin the context as used, is generally intended to convey that certainembodiments include, while other embodiments do not include, certainfeatures, elements and/or steps. Thus, such conditional language is notgenerally intended to imply that features, elements and/or steps are inany way required for one or more embodiments or that one or moreembodiments necessarily include logic for deciding, with or without userinput or prompting, whether these features, elements and/or steps areincluded or are to be performed in any particular embodiment.

Any process descriptions, elements, or blocks in the flow diagramsdescribed herein and/or depicted in the attached figures should beunderstood as potentially representing modules, segments, or portions ofcode which include one or more executable instructions for implementingspecific logical functions or steps in the process. Alternateimplementations are included within the scope of the embodimentsdescribed herein in which elements or functions may be deleted, executedout of order from that shown or discussed, including substantiallyconcurrently or in reverse order, depending on the functionalityinvolved, as would be understood by those skilled in the art.

It should be emphasized that many variations and modifications may bemade to the above-described embodiments, the elements of which are to beunderstood as being among other acceptable examples. All suchmodifications and variations are intended to be included herein withinthe scope of this disclosure and protected by the following claims.

What is claimed is:
 1. A system comprising: a non-transitory data storeincluding performance data associated with resource requests; and atleast one computing device in communication with the non-transitory datastore and having specific executable instructions operative to: identifya plurality of sets of service providers to be utilized in conjunctionwith processing a request for at least one resource, wherein individualservice providers in the plurality of sets of service providers at leastone of hosts or processes content on behalf of a content provider; forindividual sets of service providers identified in the plurality of setsof service providers: cause an initiation of a request for the at leastone resource to the individual set of service providers; and responsiveto processing the request, obtain performance data associated with theprocessing of the request using the individual set of service providers;and provide, to the content provider, an evaluation of the performancedata for the plurality of sets of services providers, the performancedata associated with processing the same at least one resource using theindividual sets of service providers.
 2. The system as recited in claim1, wherein the at least one resource corresponds to a first resource andone or more embedded resources.
 3. The system as recited in claim 1,wherein the at least one resource corresponds to an embedded resource ofanother resource.
 4. The system as recited in claim 1, whereinidentifying a plurality of service providers to be utilized inconjunction with processing a request for at least one resourcecomprises identifying a plurality of service providers selected by thecontent provider to be utilized in conjunction with processing a requestfor at least one resource.
 5. The system as recited in claim 1, whereinidentifying a plurality of service providers to be utilized inconjunction with processing a request for at least one resourcecomprises identifying a plurality of service providers to be utilized inconjunction with processing a request for at least one resource based onprior performance data corresponding to processing the at least oneresource.
 6. The system as recited in claim 1, wherein identifying aplurality of service providers to be utilized in conjunction withprocessing a request for at least one resource comprises automaticallyidentifying a plurality of service providers to be utilized inconjunction with processing a request for at least one resource withoutinput from the content provider.
 7. The system as recited in claim 1,wherein the at least one computing device is further operative todetermine, for individual sets of service providers identified in theplurality of sets of service providers, configuration information forenabling the use of the individual set of service providers to at leasthost or process the at least one resource on behalf of the contentprovider.
 8. The system as recited in claim 1, wherein causing aninitiation of a request for the at least one resource to the individualset of service providers comprises causing a client computing device toinitiate a request for the at least one resource to the individual setof service providers.
 9. The system as recited in claim 1, wherein theat least one computing device is further operative to provide arecommendation to the content provider to use a set of service providersfrom the plurality of sets of service providers to at least one of hostor process the at least one resource on behalf of the content provider.10. A computer-implemented method comprising: identifying a plurality ofsets of service providers to be utilized in conjunction with processinga request for at least one resource, wherein individual serviceproviders in the plurality of sets of service providers at least one ofhosts or processes content on behalf of a content provider; forindividual sets of service providers identified in the plurality of setsof service providers: causing an initiation of a request for the atleast one resource to the individual set of service providers; andresponsive to processing the request, obtaining performance dataassociated with the processing of the request using the individual setof service providers; and providing, to the content provider, anevaluation of the performance data for the plurality of sets of servicesproviders, the performance data associated with processing the same atleast one resource using the individual sets of service providers. 11.The computer-implemented method as recited in claim 10, whereinidentifying a plurality of service providers to be utilized inconjunction with processing a request for at least one resourcecomprises identifying a plurality of service providers selected by thecontent provider to be utilized in conjunction with processing a requestfor at least one resource.
 12. The computer-implemented method asrecited in claim 10, wherein identifying a plurality of serviceproviders to be utilized in conjunction with processing a request for atleast one resource comprises identifying a plurality of serviceproviders to be utilized in conjunction with processing a request for atleast one resource based on prior performance data corresponding toprocessing the at least one resource.
 13. The computer-implementedmethod as recited in claim 10, wherein identifying a plurality ofservice providers to be utilized in conjunction with processing arequest for at least one resource comprises automatically identifying aplurality of service providers to be utilized in conjunction withprocessing a request for at least one resource without input from thecontent provider.
 14. The computer-implemented method as recited inclaim 10 further comprising determining, for individual sets of serviceproviders identified in the plurality of sets of service providers,configuration information for enabling the use of the individual set ofservice providers to at least host or process the at least one resourceon behalf of the content provider.
 15. The computer-implemented methodas recited in claim 10, wherein causing an initiation of a request forthe at least one resource to the individual set of service providerscomprises causing a client computing device to initiate a request forthe at least one resource to the individual set of service providers.16. The computer-implemented method as recited in claim 15, wherein theperformance data corresponding to the processing of the request usingthe individual set of service providers is obtained from the clientcomputing device.
 17. The computer-implemented method as recited inclaim 15, wherein the performance data corresponding to the processingof the request using the individual set of service providers is obtainedfrom the content provider computing device.
 18. The computer-implementedmethod as recited in claim 10 further comprising providing arecommendation to the content provider to use a set of service providersfrom the plurality of sets of service providers to at least one of hostor process the at least one resource on behalf of the content provider.19. A non-transitory computer readable storage medium storing computerexecutable instructions that when executed by a processor performoperations comprising: identifying a plurality of sets of serviceproviders to be utilized in conjunction with processing a request for atleast one resource, wherein individual service providers in theplurality of sets of service providers at least one of hosts orprocesses content on behalf of a content provider; for individual setsof service providers identified in the plurality of sets of serviceproviders: causing an initiation of a request for the at least oneresource to the individual set of service providers; and responsive toprocessing the request, obtaining performance data associated with theprocessing of the request using the individual set of service providers;and providing, to the content provider, an evaluation of the performancedata for the plurality of sets of services providers, the performancedata associated with processing the same at least one resource using theindividual sets of service providers.
 20. The non-transitory computerreadable storage medium of claim 19, wherein the operations furthercomprise providing a recommendation to the content provider to use a setof service providers from the plurality of sets of service providers toat least one of host or process the at least one resource on behalf ofthe content provider.